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' Abstract 

■ We reduce a case of the hidden subgroup problem (HSP) in SL(2; q), PSL(2; q), and PGL(2; q), three 

related families of finite groups of Lie type, to efficiently solvable HSPs in the affine group AGL(l;q). 
, These groups act on projective space in an "almost" 3-transitive way, and we use this fact in each group 

to distinguish conjugates of its Borel (upper triangular) subgroup, which is also the stabilizer subgroup 
of an element of projective space. Our observation is mainly group-theoretic, and as such breaks little 
new ground in quantum algorithms. Nonetheless, these appear to be the first positive results on the HSP 
in finite simple groups such as PSL(2; q). 



1 Introduction: hidden subgroup problems 

One of the principal quantum algorithmic paradigms is the use of the Fourier transform to discover periodic- 
ities hidden in a black-box function / defined on a group. In the examples relevant to quantum computing, 
an oracle function / defined on a group G has "hidden periodicity" if there is a "hidden" subgroup H of 
G so that / is precisely invariant under translation by H or, equivalently, / is constant on the cosets of H 
and takes distinct values on distinct cosets. The hidden subgroup problem is the problem of determining the 
subgroup H (or, more generally, a short description of it, such as a generating set) from such a function. 
The standard approach is to use the oracle function / to create coset states 

' ' cGG 
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where 

Different subgroups yield different coset states, which must then be distinguished by some series of quantum 

measurements. 

For abelian subgroups, sampUng these states in the Fourier basis of the group G is sufficient to completely 
determine a hidden subgroup in an efficient manner. For nonahelian subgroups, the Fourier basis takes the 
form {1/3, where p is the name of an irreducible representation and i and j index a row and column 
in a chosen basis. Although a number of interesting results have been obtained on the nonabelian HSP, 
the groups for which efficient solutions are known remain woefully few. Friedl, Ivanyos, Magniez, Santha, 
and Sen solve a problem they call the Hidden Translation Problem, and thus generalize this further to what 
they call "smoothly solvable" groups: these are solvable groups whose derived series is of constant length 
and whose abelian factors are each the direct product of an abelian group of bounded exponent and one 
of polynomial size [4]. Moore, Rockmorc, Russell, and Schulman give an efficient algorithm for the affine 
groups AGL(l;p) = Zp x Z*, and more generally Zp x Zg where q = {p — l)/polylog(p). Bacon, Childs, 
and van Dam derive algorithms for the Heisenberg group and other "nearly abelian" groups of the form 
A x Zp, where A is abelian, by showing that the "Pretty Good Measurement" is the optimal measurement 
for distinguishing the corresponding coset states [1]. Recently, Ivanyos, Sanselme, and Santha [7, 8] give an 
efficient algorithm for the HSP in nilpotent groups of class 2. 

However, for groups of the greatest algorithmic interest, such as the symmetric groiip Sn for which 
solving the HSP would solve Graph Isomorphism, the hidden subgroup problem appears to be quite hard. 
Moore, Russell, and Schulman showed that the standard approach of Fourier sampling individual coset 
states fails [14] . Hallgren et al. showed under very general assumptions that highly-entangled measurements 
over many coset states are necessary in any sufficiently nonabelian group [6]. For Sn in particular, Moore, 
Russell and Sniady showed that the main proposal for an algorithm of this kind, a sieve approach due to 
Kuperberg [9], cannot succeed [15]. 

It is tempting to think that the difficulty of the HSP on the symmetric group is partly due to the 
appearance of the alternating group An as a subgroup. For n > 5, ^„ forms one of the families of nonabelian 
finite simple groups. All known algorithmic techniques for the HSP work by breaking the group down into 
abelian pieces, as a semidirect product or through its derived series. Since simple groups cannot be broken 
down this way, it seems that any positive results on the HSP for simple groups is potentially valuable. 

We offer a small advance in this direction. We show how to efficiently solve a restricted case of HSP 
for the family of finite simple groups PSL(2;g), and for two related finite groups of Lie type. No new 
quantum techniques are introduced; instead, we point out a group-theoretic reduction to a mild extension 
of a previously solved case of the HSP. Unfortunately, this reduction only applies to one set of subgroups, 
and there is no obvious generalization that covers the other subgroups. On the other hand, we show that a 
similar reduction works in any group which acts on some set in a sufficiently transitive way, though this is 
unhelpful in many obvious cases. 

2 Reduction 

We start with a trivial observation: suppose we have a restricted case of the hidden subgroup problem 
where we need to distinguish among a family of subgroups Hi, ...,Ht C G. If there is a subgroup F whose 
intersections Ki = HidF are distinct, then we can reduce the original hidden subgroup problem to the 
corresponding one on F, consisting of distinguishing among the Ki, by restricting the oracle to F, rather 
than the original domain G. 

The subgroups in question will be the stabilizers of one or more elements under a suitably transitive 
group action. Recall the following definitions: 

Definition 1. A group action of a group G on a set fl is a homomorphism (j) from G to the group of 
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permutations on CI. In other words, 



(I>{9i92){x) = (p{gi){(l>{92){x)) . 
When the group action is understood, we will often write just gi{x) for cf){gi){x). 

Definition 2. A transitive group action on a set Q. is one such that for any a,j3 € there is at least one 
g G G such that g{a) = 13. A fc-transitive group action is one such that any k-tuple of distinct elements 

(ai, . . . , afe) can be mapped to any k-tuple of distinct elem,ents . . . , (3^). That is, given that ai = aj and 
/3i = /3j only when i = j, there is at least one g such that g{ai) = (3i for all i = 1, . . . ,k A group is called 
k-transitive if it has a k-transitive group action on some set. 

Definition 3. Given an element a G fl, the stabilizer of a with respect to a given action by a group G is 
the subgroup Ga = {g G G \ g{a) = a}. Given a subset S C Q, the pointwise stabilizer is 

Gs = {9eG\yaGS: g{a) = a} = f| G„ . 

When S is small we will abuse notation by writing, for instance, G^ or Ga,p. 

Let's consider the case of the HSP where we wish to distinguish the one-point stabihzers Ga from each 
other. If G is transitive, these are conjugates of each other, since Gfj = gGaf)^^ for any g such that 
g{a) = p. Conversely, gGag^^ = Gg(^a)j so any conjugate of a stabilizer is a stabilizer. Similarly, for each a, 
the two-point stabilizers Ga,/3 labeled by /? arc conjugate subgroups in G^. 

Now suppose we restrict our queries to the oracle to Ga- We then get a coset state corresponding to 

Ga n G/3 = Ga,l3- 

PGa.f, = TTTT Y] \cGa,l3) {cGa,i3\ ■ 

This reduces the problem of distinguishing the one-point stabilizers Gp, as subgroups of G, to that of 
distinguishing the two-point stabilizers Ga,fi as subgroups of Ga — a potentially easier problem. Note that 
we can test for the possibility that a = (3 with a polynomial number of classical queries, since we just need 
to check that /(I) = f{g) for a set of 0(log |G|) generators of Ga- 

Of course, this whole procedure is only useful if Ga,p are distinct when Gp are distinct, or if there 
are only a (polynomially) small number of one-point stabilizers corresponding to each two-point stabilizer. 
Below we give sufficient conditions for this to be true, and use this reduction to give an explicit algorithm 
for distinguishing conjugates of the Borel subgroups in some finite groups of Lie type, including the finite 
simple groups PSL(2; q). Using the transitivity of the group action we can bound the size of these stabilizers 
relative to each other and to the original group, and hence show that they are distinct. 

Lemma 1. Suppose G has a k-transitive group action on a set Q. where = s. Then for any j < k, if 
S Cil and \S\ = j, we have 

\G\ ^ s\ 

\Gs\ {s-jy.- 

In particular, 

I |G| |„ |_ |G| ,_ |G| 



Proof. The index of Gs in G is the number of cosets. There is one coset for each j-tuple to which we can 
map S, and since G is j-transitive this includes all s!/(s — j)! ordered j-tuples. □ 

For groups that are at least 3-transitive, the following then holds: the intersection of two subgroups that 
are single-point stabilizers of il has size l/(s — 1) of both of the subgroups. The intersection with a third 
stabilizer subgroup is 1/ (s — 2) this size again. In particular, this means that when subgroups G/j and G^ are 
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distinct, then their intersections GaOGp = Ga,^ and Gc« HG-y = Ga,-y are distinct, because their intersection 
Ga,fi n Ga,^ = Go,,/3,7 is Smaller than either. 

In fact, we don't need full 3-transitivity for this argument to hold. The crucial fact we used was that the 
number of cosets of Ga,0,-y was greater than Ga,^ or Ga,-y Consider the following definition: 

Definition 4. A group is almost A'-transitive if there is a constant b such that G has an action on a set 
fl which is {k — 1) -transitive, and such that we can map any k-tuple of distinct elements (ai, . . . ,afc) to at 
least a fraction b of all ordered k-tuples (/3i, . . . , f3k) of distinct elements. 

Strictly speaking, there is a different notion of "almost" for different values of b. Obviously, for any group 
there is some value of b low enough that this definition applies. However, by fixing b and considering a family 
of groups we still have a useful concept. 

As an example, a group action is k-homogeneous if any set of points of size k can be mapped (setwise) 
to any other set of the same size. Since this means that any ordered A;-tuple can be mapped to at least 1/fc! 
of the ordered fc-tuples, and since all fc-homogeneous group actions are (A: — l)-transitive [3], a group with 
such an action is almost fc- transitive with b = l/k\ (in fact, with b = 

Applying the above argument to almost 3-transitive groups shows that the stabilizer of 3 distinct elements 
is smaller than the stabilizer of 2 distinct elements by a factor of (s — 2)6. So long as 6 > l/(s — 2), two-point 
stabilizers of distinct elements will be distinct. In the group families we cover, 6 = 1/2, and s grows. 



3 Families of transitive groups 

Which families of groiips and subgroups have the kind of transitivity that let us take advantage of this idea? 

Unfortunately, not many do. We can categorize based on faithful group actions, i.e., those that do not map 
any group clement other than the identity to the trivial action. Any non-faithful groiip action corresponds to 
a faithful action of a quotient of the group. Even the requirement of 2-transitivity in faithful group actions 
restricts the choices to a few sporadic groups, or one of eight infinite families [3]: The symmetric group 5„, 
the alternating group , and six different families of groups of Lie type. 

Obviously the symmetric group Sn is n-transitive, and the alternating group A„ is almost n-transitive. 
However, the size n of the set these groups act on is only polynomially large (i.e., polylogarithmic in the size 
of the groups) so we can distinguish the one-point stabilizers with a polynomial number of classical queries. 

The other infinite families are finite groups of Lie type which are defined in terms of matrices over 
finite fields subject to some conditions. These groups have natural actions by matrix multiplication 
on column vectors, or on equivalence classes of column vectors. The actions of most of these groups are 
rather complicated to describe; for more details, see [3, §7.7]. Of these, two are 3-transitive: PSL(2;g), and 
AGL(rf;2). 

There are also a number of sporadic finite groups that are up to 5-transitive, such as the Matthieu groups 
-^11) Afi2j M22^ M23, M24, built on finite geometries. However, an interesting fact is that if a group action 
has a threshold of transitivity, then it contains all permutations, or at least all even ones: for fc > 5, all finite 
groups with a fc-transitive action on a set of size n must contain A„ [3] . 



4 PSL(2;g) and some relatives 

The most interesting family of simple groups with a faithful almost 3-transitive group action is PSL(2;(7). 
To discuss it, consider instead GL(2; q), the group of invertible 2x2 matrices with entries in the finite field 
¥q, where g = is the power of some prime p. Its elements are of the form 




where a, fi,j,S G F^, and aS — /S'^ 0. We will assume that q is odd; some details change when it is a power 
of 2, but the basic results still hold. 
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A little thought reveals that |GL(2;q)| ^ {(^ - \){(^ (g + 1)<?(<? - 1)^- The subgroup SL(2; g) 

consists of the matrices with determinant 1, so |SL(2;q')| = (g + l)g(g — 1). If we take the quotient of 
these groups by the normal subgroup consisting of the scalar matrices, we obtain VGL{2\q) and PSL(2;g) 
respectively. For SL(2; q) the only scalar matrices are ±1, so |PSL(2; = + 1)<7(<7 — l)/2. 

GL(2;g) and SL(2;q') act naturally on nonzero 2-dimensional vectors. For PGL(2;q') and PSL(2;(7), we 
must identify vectors which are scalar multiples. This identification turns — {0, 0} into the projective line 

PFq. Each element of PF, corresponds to a "slope" of a vector: the vector slope cc/y, i.e., xy~'^ if 

y ^ and oo if = 0. Thus we can think of PFg as F^ U {oo}, and it has + 1 elements. 
The action of PGL(2; q) and PSL(2; q) on PF, is given by 

a. /3\ / x\ _ /ax + f3y 
7 \yj ~ yjx + Sy 

This fractional linear transformation is analogous to the Mobius transformation defined by PGL2(C): 

^a /3\ /a;\ ax + 



7 ^ J \y J ^x + 5y ' 

which can map any 3 points in the complex projective line PC (i.e., the complex plane augmented by the 

point at infinity, or the Ricmann sphere) to any other 3 points. When we replace C with the finite field Fg, the 
action of PGL(2; q) remains 3-transitive. The action of PSL(2; q) is 2-transitive, but cannot be 3-transitive, 
since there are half as many elements as there are 3-tuples. However, PSL(2; q) is almost 3-transitive in the 
sense defined above with b = 1/2, since 1/2 of all 3-tuples can be reached. SL(2; q) is also almost 3-transitivo: 
from a given tuple, it reaches the same set of tuples as PSh{2;q), with each tuple being hit twice. As a 
result, this action is obviously not faithful, for the kernel is ±1. 

Let G = PGL(2; q), and consider the one-point stabilizer subgroups of its action on PF^. A natural one 
is the Borel subgroup B of upper-triangular matrices. Such matrices preserve the set of vectors of the form 
x\ 

Q j , SO we can write B = Goo- There are q + 1 conjugates of B, including itself, one for each element of 
PFg. For instance, if we conjugate by the Weyl element ^ = > we get wBw~^ = Go, the subgroup 

of lower-triangular matrices, which preserves the set of vectors of the form 

5 An efficient algorithm for distinguishing the conjugates of the 
Borel subgroup 

Now consider the case of the HSP on these groups where the hidden subgroup is one of B's conjugates, or 
equivalently, one of the one-point stabilizers Gg- As discussed above, we solve this by restricting the oracle 
to B, and distinguishing the two-point stabilizer subgroups B Gs ^ Gs.oc as subgroups of B. To do this, 
we need to describe the structure of B explicitly. For all three families of matrix groups we discuss, namely 
SL(2; q), PSL(2; q), and PGL(2; q), B is closely related to the affine group. 

In PGL(2;g') a generic representative of B can be written ^ ^ , a ^ 0, so |B| = q{q — 1). This is 

exactly the affine group AGL(1; g') = F^ x F*. To see this, recall that AGL(1; q) consists of the set of affine 
functions on ¥q of the form x i-^ ax + /3 under composition. Now consider B's action on PFg — {oo}, which 
we (re)identify with Fg. For PGL{2;q), we have 

a /3\ fx\ __ fax + 
1 j U j " I 1 
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Obviously these elements compose as AGL(1; q), so B = AGL(1; q). 

The cases of SL(2; q) and PSL(2; q) are more complicated. The unit determinant requirement limits B 

to elements of the form q,-!^ " Thus \B\ = q{q — 1) again in SL(2; q). For PSL(2; q) we identify a with 

-a, so \B\=q{q-l)/2. 

For SL(2; q), we can enumerate the elements as: 



a 



a 



-1 



Composing two such elements gives us: 



a a-^^\ /7 7-M\ _ /a7 a-^7-^^ + 7-^aA _ /a7 {a-^j-'^){(3 + a'^d) 
J \0 7" V ~ V a-^7-^ / V a-^-i 

Here, we still have a semidirect product of the groups ¥q and F*. Unlike the affine group, where the 
multiplicative group acts directly as an automorphism on the additive group by multiplication, it instead acts 
"doubly" by multiplying twice, analogous to the "<?-hedral" groups in [13] (with q = p/2, in their notation). 
Finally, PSL(2; q) merely forgets the difference between ±a. This quotient group of SL(2; q) can also be seen 
as a subgroup of the afHne group that can only multiply by the square elements. 

In all three cases the HSP on B can be solved efficiently using small generalizations of the algorithms 
of [13]. We need to generalize slightly as [13] deals only with the case of Z„ x Fp with p prime — not 
a prime power q = p^, as here. The basic methods remain effective, though we construct and analyze a 
slightly different final measurement. The number and size of the representations remains the same (with q 
replacing p), and the methods for constructing Gelf'and-Tsetlin adapted bases are similar. As this has not 
been published in the literature, we describe the details more fully in the next section, though only what is 
necessary for our purposes. 



6 Generalizing the affine group to the prime power case 

Although there can be more types of subgroups than the ones covered in [13], we are only concerned about 
one particular type whose analog was covered there: H = (a, 0) and its conjugates = (1, 6)iJ(l, — 6), 
stabilizing the finite field element b. The representation theory is analogous, with q — \ one-dimensional 
representations (characters) depending only on a. As in the prime case, we have q conjugacy classes: the 
identity, all pure translations, and each multiplication by a different a, combined with all translations. This 
leaves us with one {q — 1) dimensional representation, p. 
In the prime case we had: 

P{{a,h))j,k=Q ^T''^ (j,fceF„^0). 
I otherwise 

where ojp = exp{2m/p). The roots of unity are the non-trivial additive characters of F^, indexed by j, 
evaluated at b. We can extend this to the prime power case simply by replacing bj, with b ■ j 



b-j = Trbj = TrF^„/F^ bj = J2 (bj) 



which as b varies, exactly covers the full set of non-trivial linear operators from Fpn to Fj,, and ui^'^ ex- 
actly covers the set of additive characters. Performing weak measurement on the coset state yields p with 
probability P{p) = 1 — 1/q. Conditioned on that outcome, we get the following projection operator: 

7rH<.(p),,fc = ^<(^-^^ 
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As in [13], wc then pciiform a Fourier transform on the rows, and ignore the eolumns. There they 
performed the Fourier transform over Zp_i, as there were p—1 rows. However, the structure for general q is 
not Z* = Zq_i, but F*. The interaction we want to capture is the additive one, not the multiplicative one. 
We can still perform the abelian transform over the additive group Fg = Z^ — the zero component we lack 
is, of course, zero. The probability of observing a frequency £ G Z" is then: 



P{1) = 



1 



q{q-l) 
1 

a(a-l) 



-1 + qSib\ 



i = b 



For the case of B in PSL(2;g), we can analyze the equivalent measurements via the embedding in the 
full afRne group, just as in the prime case. Let o be a generator of the "even" multiplicative subgroup of 
F*. c;onsisting of elements that are squares. is then elemcints of the form (a*, (1 — a*)6) stabilizing b. For 
these subgroups, the trivial representation, a "sign" representation, and the large representation occur with 
non-zero probability. The first two have vanishingly small probability, 0{l/q). 

In the following we use the notation G{m, a) = X^j-g^. m{x)a{x) for the Gauss sum of a multiplicative 

and an additive character, where Xk{j) = "^p is the additive character of F^ with frequency k E Z^. We 
follow the common convention that non-trivial multiplicative characters vanish at 0. We use the quadratic 
character rj of F*, which is 1 for squares, and —1 for non-squares, to select rows and columns which differ 
by values in the "even" subgroup mentioned above. 

Weak measurement gives us the representation p with overwhelming probability. Conditioning on this 
event, we get the mixed state 



v/2 



9-1/2 



Measuring the column k gives us, up to a phase, p{b)j — \J -^pi '^^ "' (1 ± v{j))/2- 

We again include the zero component, with zero weight, and perform the abelian Fourier transform over 
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the additive group Fg = Z^. The probabiUty of measuring frequency i is 



m = - 

q 



2 


q{q- 


1) 


2 






1) 


1 




2q{q- 
1 


-1) 


2q{q- 
1 


-1) 


2q{q- 


-1) 



j¥0 



X] X6-^ (i) ± X) Xb-eij)v{j) 
|G(1,X6-£)±G(77,X6-^)|' 



where d is odd for odd n if = 3 (mod 4), and d is even otherwise. 

For ^ = 6 we have P(£) = {q - if /2q{q - 1) = {q - l)/2q. For £ ^ & we have P(f) = (g + l)/4g(g - 1) 
if d is odd. If £ ^ 6 and is even, we have P{() — {q± 2g^/^ + l)/4,q{q — 1). In any case, the probability of 
observing h is 



q-1 
2q 



■0{l/q), 



so repeating this measurement will allow us to identify £ = b with any desired probability. As SL(2; q) is a 
small extension of PSL(2; q), we can handle it similarly, by Theorem 8 in [13]. 



7 AGL^d; 2) and its stabilizer subgroups 

An interesting question is whether it is useful to apply this approach to the other family of 3-transitive 
groups. This is the d-dimensional affine group AGL(d; 2), consisting of functions on of the form Av + B, 

where A e GLd(F2) and B € It can be expressed as a block matrix of the form ^ . It is the 

semidirect product GL{d; 2) k Fj, and hence obviously not simple. That it is triply transitive can be seen 
by realizing that the affine geometry it acts on has no three points that are collinear. 

The stabilizer subgroups are 2^^ conjugate subgroups of the original GL{d;2). Obviously this stabilizes 
the point 0, and is the largest subgroup that will, as GL{d; 2) has two orbits: the zero vector, and all others. 
A general point P is stabilized by translating it to with the element {A, B) = (1, P), applying any element 
of GL((Z;2), and then translating back. To apply our method we need to look at the intersections. 

Consider the point 1 = (0, 0, 1)-^. Splitting A into two diagonal blocks of size (d — 1) x (rf — 1) and 
1x1 and two off-diagonal blocks of size (d — 1) x 1 and 1 x (d — 1) allows us to see that 1 is stabilized by a 
(transposed) copy of AGL{d — 1; 2) living in GL{d; 2). The last column must be 1 = (0, 0, 1)^ to preserve 
1. The large {d — 1) x {d— 1) block must be in GL(d;2) to keep the the entire transformation invertible, 
and anything in GL{d; 2) will preserve the first d — 1 bits of 1. The rest of the last row can be arbitrary, 
resulting in a subgroup isomorphic to AGL{d — 1; 2). 

As a result, distinguishing the stabilizers of points reduces to distinguishing conjugates of a smaller 
transposed copy of the affine group in the general linear group. This last reduction does not immediately 
yield an efficient new quantum algorithm. 
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8 Conclusion 



It is interesting to note that although we can Fourier sample over AGL(d;2) efficiently [12], we don't know 
how to do so in the projective groups. The fastest known classical Fourier transform for SL(2; q) or PSL(2; q) 
takes ld{q'^\ogq) time [10], and the natural quantum adaptation of this takes Q{q\ogq) time [12]. If q is 
exponentially large, this is polynomial, rather than polylogarithmic, in the size of the group. In the absence 
of new techniques for the FFT or QFT, this suggests that we need to somehow reduce the HSP in PSL(2; q) 
to that in some smaller, simpler group — which was the original motivation for our work. 

We conclude by asking whether our analysis of AGL(d; 2) can be extended to give an efficient algorithm 
distinguishing its stabilizer subgroups, or whether any of the other 2-transitive groups have usable "almost" 

3- transitive actions. 
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